Cross-Platform Resource Scheduling for Spark and MapReduce on YARN
نویسندگان
چکیده
منابع مشابه
Research of Decision Tree on YARN Using MapReduce and Spark
Decision tree is one of the most widely used classification methods. For massive data processing, MapReduce is a good choice. Whereas, MapReduce is not suitable for iterative algorithms. The programming model of Spark is proposed as a memory-based framework that is fit for iterative algorithms and interactive data mining. In this paper, C4.5 is implemented on both MapReduce and Spark. The resul...
متن کاملResource-Aware Adaptive Scheduling for MapReduce Clusters
We present a resource-aware scheduling technique for MapReduce multi-job workloads that aims at improving resource utilization across machines while observing completion time goals. Existing MapReduce schedulers define a static number of slots to represent the capacity of a cluster, creating a fixed number of execution slots per machine. This abstraction works for homogeneous workloads, but fai...
متن کاملEvaluation of Parallel Differential Evolution Implementations on MapReduce and Spark
Global optimization problems arise in many areas of science and engineering, computational and systems biology and bioinformatics among them. Many research efforts have focused on developing parallel metaheuristics to solve them in reasonable computation times. Recently, new programming models are being proposed to deal with large scale computations on commodity clusters and Cloud resources. In...
متن کاملA Review: Mapreduce and Spark for Big Data Analytics
In this paper we discuss the various challenges of Big Data and problem arises due to continuous explosion of data resulting from the likes of social media and other online sources to gain access to deeper analysis of their data. This paper discusses two of the comparison of Hadoop Map Reduce and the recently introduced Apache Spark – both of which provide a processing model for analyzing big d...
متن کاملOptimal resource allocation and scheduling for the CELL BE platform
Resource allocation and scheduling for multicore platforms is one of the most critical challenges in today’s embedded computing. In this paper we focus on a well-known multicore platform, namely the Cell BE processor, and we address the problem of allocating and scheduling its processors, communication channels and memories, with the goal of minimizing execution time for complex data streaming ...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: IEEE Transactions on Computers
سال: 2017
ISSN: 0018-9340
DOI: 10.1109/tc.2017.2669964